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SPECIFICATION 



TO ALL WHOM IT MAY CONCERN: 



BE IT KNOWN, that I, Michael J. Freeman, Ph.D., and Craig Ullman, citizens 

of the United States and residents of the State of New York, have invented certain 

new and useful improvements in: 

A DIGITAL INTERACTIVE SYSTEM FOR PROVIDING FULL INTERACTIVITY 

WITH LIVE PROGRAMMING EVENTS 

of which the following is a specification. 
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A DIGITAL INTERACTIVE SYSTEM FOR PROVIDING FULL INTERACTIVITY 

WITH LIVE PROGRAMMING EVENTS 

RELATED APPLICATIONS 

This is a continuation-in-part of application Serial No. 08/598,382, filed 
February 8, 1996, which is a continuation-in-part of application Serial No. 
08/ 443,607, filed May 18, 1995, which is continuation-in-part of application Serial 
No. 08/166,608, filed December 13, 1993, now abandoned, which in turn is a 
continuation of application Serial No. 07/797,298, filed November 25, 1991, now 
abandoned. 

BACKGROUND OF THE INVENTION 

Interactive video and audio presentation systems are currently being 
introduced into the entertainment and educational industries. A prominent 
interactive technology that has been applied successfully in these industries is based 
- on providing interactivity in a one-way system through the provision of multiple 
time-synched parallel channels of information. For example, commonly owned 
Freeman et al. patents, U.S. patent nos. 4,264,925 and 4,264,924, which provide both 
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audio and video interactivity, disclose interactive television systems where 

switching among multiple broadcast or cable channels based on viewer selections 

provides an interactive capability. 

These systems have been enhanced to include memory fimctions using 

computer logic and memory, where selection of system responses played to the 

viewer are based on the processing and storage of subscriber responses, as disclosed 

in Freeman patent, U.S. patent no. 4,507,680. 

The benefits of providing interactivity through the use of different audio 

responses is disclosed in Freeman, U.S. patent nos. 4,847,698, 4,847,699 and 4,847,700. 

These television systems provide a common video signal accompanied by several 
synchronized audio charmels to provide content related user selectable responses. 
The audio signals produce different audio responses, and in some cases, these are 
syllable synched to a first audio script and to the video signal (such as to a person or 
character on a display), providing the perception that the person's or character's 
mouth movements match the spoken words. 

Interactivity is brought to the classroom in the Freeman U.S. patent no. 
5,537,141. The distance learning system claimed in this application enhances the 
classroom educational experience through an innovative use of interactive 
technology over transmission independent media. When an instructor, either 
broadcast live on video or displayed from videotape, asks a question, each and every 
student responds, preferably by entering a response on a remote handset, and each 
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student immediately receives a distinct and substantive audio response to his or her 
unique selection. The individualization of audio response from the interactive 
program is a major aspect of the invention. 

Individualization of audio is brought to the home based on the technology 
disclosed in Freeman U.S. patent no. 5,585,858. This system provides a program that 
can be watched on any conventional television set or multimedia computer as a 
normal program. But if the viewer has a special interactive program box connected 
to the television, he or she can experience a fully functional interactive program. 
Each interactive viewer enjoys personalized audio responses and video graphics 
overlayed on the screen. The interactive program can be provided to television sets 
or to computers by cable, direct broadcast satellite, television broadcast or other 
transmission means, and can be analog or digital. 

However, what is needed is an interactive presentation system for providing 
true video, audio and graphics interactivity with digital programs broadcast live. 
Such a system must efficiently package all the digital elements of the live interactive 
program at a centralized control studio and allow viewers at home to receive 
personalized interactive programming. 

SUMMARY OF THE INVENTION 

In accordance with the invention, there is provided an interactive digital 
system allowing the viewer active participation in selecting digital video streams. 
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associated with different camera angles, for example, and integrated audio and /or 
graphics segments. Further, Web pages from Internet Web sites can be integrated 
into the program. The invention is particularly suited for the environment of live 
events, such as the broadcast of live sporting events. The viewer can appear to 
direct the camera shots by instantly changing among various camera angles, choose 
player interviews, or display associated statistical data on the team or players via 
graphics. In this manner, the system allows the individual subscriber to act as if he 
or she has control over how the program is directed and presented on their personal 
television set. In addition to selecting different camera angles, various audio 
options, closeups, slow motion, replays, graphics overlays, graphics or audio from 
Web sites, etc., are all possible. Further, games can be integrated with the live sports 
programming to increase viewer interest. 

Thus, viewers can customize the content of programs. The interactive digital 
programming of the present invention is particularly advantageous for viewing 
live sporting events. Viewers are not limited to selecting from multiple camera 
angles, but may also call up player statistics on demand, listen to selected player 
interviews, etc. Cameras can be focused on different segments of an event. Further, 
video options could include video replay, slow motion effects, isolation on a 
particular player or group, etc. Changes are seamless, thereby adding to the effect 
that the viewer is directing the television show just as a director now does from a 
control room. 
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This "director" role by the viewer is possible due to the interactive technology 
of the present invention and also due to the digital compression and transmission 
scheme which allows for much greater information throughput over a given 
bandwidth, allowing viewers to choose from angles that are already available but 
presently cut by the director. 

The digital interactive system is based upon branches which occur in the 
course of the full-motion video. Branches are real-time parallel paths that may be 
other full-motion video segments, graphics which are integrated into the video, 
audio segments, and /or retrieved Web pages which are integrated into the live 
event. 

Sometimes, the interactive digital system will act upon the viewer's response 
immediately; other times, it will utilize ACTV's imique "profiling" concept to act 
upon the response later. This technology enables the system to "remember" the 
viewer's responses and desires, and integrate them into the video, audio, graphics 
and/ or Web site information at a later point. For example, the viewer could specify 
at the beginning of a football game to isolate the offensive quarterback of a particular 
team. Thus, whenever the team of choice is on offense, the video isolation of the 
quarterback is displayed to the viewer automatically. Or, based on how a viewer has 
selected camera angles, replays, etc., over the past five minutes, the system acts to 
mimic these selections at later times during the program. The system of the present 
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invention 'learns'' from the viewer how they want to view the game, and thus, 
continues viewer selection sequences made earlier. 

At the source, the present invention comprises a plurality of video cameras, 
each of the video cameras relaying a different predetermined view of an event. The 
video signals corresponding to the different cameras are forwarded to a central 
control studio. Further, one or more audio signals or graphic statistical overlays can 
be collected and sent to a central control studio. After receiving the video, audio, 
and graphics signals at the central control studio, these signals are digitized and 
compressed in digital video and audio compressors. These signals are then 
combined with special data codes into a "digital package," and subsequently, 
transmitted over a cable distribution system. The special data codes are the keys to 
imlocking the interactive potential of the program. 

The digital program signals are transmitted to a receive site by any suitable 
transmission means. Once received by a receive antenna, the digital program 
signals are passed along on a digital cable television distribution system to the 
viewer homes. Further, some other signals or commercials can be inserted at the 
local head end. The signals are received and processed in a digital cable box. 
Selections of the video, audio, graphics display and /or Web pages can be made as a 
function of immediate viewer entries, or to interrogatory responses presented at the 
beginning or during the program, or based on a prestored viewer profile. Once a 
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decision is made to switch from one video option to another video option, the 
digital switch is performed seamlessly. 

As mentioned above, the program at predetermined times or immediately 
upon user entry can retrieve and branch to informative segments from Web sites. 
For example, a viewer watching a sporting event, through the system of the present 
invention, can receive a stream of Web pages which provide additional, specific 
information relating to a favorite player, team or perhaps the remaining schedule 
for the sports team, as examples. In addition, users can take advantage of the two- 
way capabilities of the Internet to respond to polls or to link to additional sites. 

Another Internet-based application allows advertisers to speak more directly 
to consumers by directly sending Web pages to the consumer instead of merely 
displaying Web addresses in their commercials. The particular advertising 
information from Web sites can be targeted to viewers based on the viewer profile, 
stored either in the digital set top box or at the cable headend. Alternatively, Web 
site access can be initiated by the viewer by simply clicking on the remote during the 
commercial. Thus, viewers have the capability to individually select Web sites if 
they want more information from advertisers, for example. 

The video programming and corresponding Internet pages can be viewed 
either on personal computers equipped with a television card on special digital cable 
boxes with stored interactive Internet application software providing Internet access. 
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or on digital television sets, all of which would utilize the specialized TV/Internet 
software of the present invention. 

The present invention also has applications for other types of programming. 
For example, viewers can direct the scenes of a murder mystery. Switching from 
one scene to another can be done seamlessly without noticeable effect on the viewer. 
Further, the present invention can be used for any kind of live or pre-recorded 
event. For instance, a music concert or a political convention can be enhanced in 
the manner of the invention. 

Accordingly, a primary objective of this invention is providing an enhanced 
digital live program allowing the display to be tailored to the user's desires, choices 
or interests. 

It is an object of this invention to personalize and enhance live sporting 
events for the viewer. 

DESCRIPTION OF THE DRAWINGS 

Figure 1 is a diagram of the network and equipment for providing live digital 
programming. 

Figure 2 is a block diagram of an interactive digital cable box allowing 
seamless switching between video signals. 

Figure 3 is a block diagram of an alternative dual-tuner interactive digital 
cable box allowing seamless switching between video signals. 
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Figure 4 is a block diagram of another alternative interactive digital cable box 
allowing seamless switching between video signals. 

Figure 5 is a time diagram showing a representation of trigger points and 
corresponding video, audio and/or graphics segments, one or a combination of 
which are selected for presentation to the subscriber immediately after the execution 
of the trigger point function. 

Figure 6 is a block diagram of an alternative embodiment of the interactive 
system including Internet access. 

Figure 7 is a block diagram of the two-way configuration of the system. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The present invention is an interactive digital system 1 for producing a 
powerful personalized program allowing the home viewer an expanded set of 
programming options. Digital TV streams are put into digital packages made up of 
video, audio, data codes and graphics, and are used to provide personalized 
responses to viewer selections. Such responses can be further enhanced by allowing 
access to Internet Web sites 170. In this manner, sports such as golf, football, 
baseball, basketball, etc. can now be watched with greater interest and involvement. 
However, even further enhanced interactivity is possible with the present 
invention due to the provision of various profiling and memory features. 
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As shown in Figure 1, the present invention begins with the gathering of 
several possible video streams by way of cameras 100, strategically located at a 
sporting event 10, for example. Currently, many cameras 100 are employed at a 
sporting event 10, Super Bowl coverage, for example, typically encompasses 25 to 30 
cameras. These live video streams can be integrated with recorded video streams 
which, for example, could include highlights from the current game or past games, 
player profiles, etc. To describe the components and operation of the present 
invention, the production and transmission of a live sporting event 10, i.e., football, 
is chosen to present the invention features. However, other applications can be 
appreciated by the reader, including several disclosed below. 

A. The Control Studio 

The interactive broadcast program is prepared at the control studio 5 into 
digital packages. The control studio 5, as shown in Figure 1, allows a producer to 
create and introduce interactive elements during a live broadcast. In one preferred 
embodiment, the producer pre-records a set of interrogatories or instructions for the 
user. These interrogatories may include such questions as the following: 

SELECT THE CAMERA ANGLES /OPTIONS THAT YOU PREFER: 

OPTIONS: END-ZONE 

FIFTY YARD LINE 

FOCUS ON THE QUARTERBACK 

FOCUS ON THE DEFENSE 
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FOCUS ON THE CHEERLEADERS 
PROVIDE HIGHLIGHTS 

WHO IS YOUR FAVORITE TEAM IN THE GAME? 

OPTIONS: BLUE TEAM 
RED TEAM 

WHO IS YOUR FAVORITE PLAYER IN THE GAME? 

OPTIONS: RED RUNNER 

BLUE QUARTERBACK 
BLUE DEFENSIVE END 
RED QUARTERBACK 
ETC. 

ARE YOU INTERESTED IN RECEIVING STATISTICAL SUMMARIES? 

WOULD YOU ENJOY LISTENING TO PLAYER INTERVIEWS DURING THE 
GAME? 

Such interrogatories can be presented to the viewer at the beginning of the 
broadcast or scattered throughout the program. Interactive responses to such 
interrogatories would include video, audio and graphics personalized to the 
particular viewer. The preparation of the graphics for presenting such 
interrogatories occurs off-line at the control studio 5 using chyron or any graphics 
language. These interrogatories or instructions will ultimately be displayed to the 
' home viewer, preferably in the form of graphics, to facilitate the interactive 
responses. The producer creates these graphic video slides of questions on a 
computer using the text editor and chyron. Associated with each question, the 
producer enters a number of possible viewer options. Then, the producer relates 
each possible user entry to one or more corresponding interactive responses. If the 



11 



Docket No. 5038 



response is information from an Internet Web page, the producer will indicate the 
particular Uniform Resource Locator (URL) of the Web page. The producer sets a 
time stamp for when each particular question will appear during the program. As 
explained below in more detail, the viewer response to a query will be used to direct 
which video (and /or audio, graphics, or Web page) option will be provided to the 
viewer. In the present invention, the interactive response to the query can occur 
immediately following the entry of the viewer entry or at some predetermined later 
time in the program using "trigger points," 500 as explained in detail below. 

With respect to the video segments, cameras 100 are preferably trained on 
different segments of the sporting event 10. As is common with broadcasts of a 
football game, for example, cameras 100 could be located in the endzone, press box, 
the field and at various other locations throughout the stadium. Further, various 
video options can be created including video replay, slow motion, isolation on 
cheerleaders, particular player or group of players. Instant replays are created by 
delaying the live feed for a certain number of seconds. These video streams are sent 
to a control studio 5. The control studio 5 contains the necessary equipment for 
packaging the program for delivery to the viewers. The studio 5 contains a video 
switcher 105 which receives the live signals from the cameras 100 by way of various 
input lines. Further, lines carrying recorded video streams from one or more VCRs 
110 , computers or CD players feed into the video switcher 105. The video switcher 
105 also receives video inputs from the control computer 135. Further, various 
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graphics screens, depicting, for example, sports team or player statistics can be 
designed with the control personal computer 135 and forwarded to the digital video 
switch 105. The producer, via the control PC 135, directs which video options to pass 
through the video switcher 105. At the output of the video switcher 105, each of the 
different output video streams access a separate encoder 125 and are all 
GENLOCKED, so that each video stream is synchronized with the other video 
streams. 

After encoding, the video streams are input into a video compressor 125. 
Preferably, the digital compression scheme is MPEG-2, Preferably, 64 Quadrature 
Amplitude Modulation (QAM) is used as the modulation scheme. In this digital 
embodiment, four channels of digitally-compressed video content would carry 
about 27 Mbps using 6 MHz of bandwidth. Alternatively, if 256 QAM is employed, 
seven-to-one video compression can be achieved with the MPEG-2 scheme. While 
MPEG-2 is the preferred compression scheme, the signals can be compressed 
according to any known standard including MPEG-1, JPEG, or other DCT coding 
scheme, wavelets, fractals or other transform or waveform based technique. 

The control studio 5 also contains an audio switcher 115 which receives live 
audio signals from microphones or recorded audio from tape players 120, CDs, VCRs 
110, etc.. The control computer 135 sends commands to the audio switcher 115 
directing which audio options should pass through the switch 115. Further, in the 
audio switcher 115 the various audio signals can be aligned to match the various 
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video signals in time. In addition, VCR audio output is received by the audio 
switcher 115. The present invention can accommodate any number of audio signals 
as output from the audio switcher 115, as directed by the producer. The audio 
outputs are received by an digital audio encoder/compressor 130. The audio signals 
are then preferably sampled, encoded and compressed in the digital audio 
encoder/compressor 130. The encoding technique can be a waveform coding 
technique such as PCM, ADPCM or DM. Alternatively, the signals can be encoded 
using synthesizer or vocoder techniques such as MUSICAM, Linear Predictive 
Coding (LPC), Adaptive Predictive Coding (APC), and Sub-band coding. Generally, 
the transmission rate is about 256 kbps per audio for the stereo pair. 

The timing and control for integrating the various multimedia elements is 
provided by the ACTV authoring language, a imique set of interactive data codes to 
facilitate the interactive process. The data codes are stored in memory in the control 
computer 135 as part of the ACTV programming language. The codes comprise 
commands, or branch codes, for branching between interactive options, timing 
signals for controlling the interactive program, data or text, commands for 
termination and initiation or interactive program viewing, or triggers for executing 
macros. Preferably, these commands are output from the control computer 135 and 
multiplexed with the video streams in the MPEG-2 compressor 125, as shown in 
Figure 1. Interactive options that can be branched to based on the branch codes 
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include video segments, audio segments, graphics segments and /or identified Web 
pages. 

There are several commands in the ACTV authoring language that function 
to provide the interactive sports programming applications. These commands are 
explained below in Section D, which details programming applications. 

B. The Transmission System 

As shown in figure 1, the digital interactive system 1 uses an interactive 
program delivery system with any transmission means including satellite 15, cable 
150, wire or television broadcast 175 to deliver the interactive program (hereinafter 
"composite interactive program'O from the control studio 5 for distribution to 
subscribers in their homes. At the control studio 5, the signals from the digital 
multiplexer 140 are converted to RF and distributed to a microwave 175, cable 150 or 
satellite 15 network. Preferably, the digital interactive signal is forwarded from the 
control studio 5 to a cable headend 150, and subsequently, sent to the homes via the 
cable network. 

The program is preferably the broadcast of a live event. For example, live 
sporting events with added interactive elements can be broadcast from the control 
studio 5. Such live interactive elements could be different camera angles 100, slow 
motion video, etc., as discussed above, while also incorporating prerecorded 
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interactive segments such as highlights. Alternatively, the prograra can be 
produced off-line and stored in a program storage means at the control studio 5. 

In a satellite broadcast transmission, the digital interactive signals are 
transmitted to uplink equipment where they may be multiplexed, upconverted, 
modulated, ampHfied and transmitted by satellite 15 to the receiver site 155 for 
distribution to the homes. 

At the reception end, the composite digital interactive signals enter a receiver 
155 where the signals are demultiplexed, downconverted, demodulated and then 
passed to a cable distribution system that directs the signals to the homes. Although 
a cable distribution system 150 is the preferred transmission media to the homes, the 
digital signals may also be distributed by any conventionally known technique 
including satellite 15 to digital satellite receivers 155 at the home, fiberoptics, low or 
high power broadcast television 175, telephone lines, cellular networks, and similar 
technology can be used interchangeably with this program delivery system. 

C. The Interactive Digital Box 

The interactive digital box 25 is shown schematically in Figure 2. Preferably, 
the interactive digital box is a specially adapted digital cable box 25. The controller 
260 determines what video, audio, graphics and/or Web pages to display based upon 
the interactive commands which it receives. Based upon the commands, it plays 
the appropriate video, audio, graphics or Web page options. The graphics can either 
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be created and sent from the control studio 5 or the graphical images can be created 
at the interactive digital box 25 based on instructions preferably in the interactive 
commands. The interactive digital box 25 cormects to a television 165 or other 
display monitor. Further, the interactive digital box 25 can be connected to a digital 
television 195, in which case an RF modulator 245 is not necessary. Each 
downstream transmission reaches the subscriber's house, shown in Figure 2, 
preferably through a tap and drop cable. 

The user interacts with the program through the input device 20. Preferably, 
the input device 20 is a typical television remote. The user interface 270 may be an 
infrared, wireless, or wired receiver that receives information from the input device 
20, 

Regardless of the type of input device 20, user inputs can be utilized by the 
present invention immediately, or at a later time, to result in personalized graphics, 
video and /or audio presentation. For example, the present invention utilizes 
"trigger points,'' 500 as described below, to enable subsequent branches among 
multimedia segments during the show. 

Certain commands are sent from the control studio 5 as part of the digital 
interactive programming to facilitate the collection of user entries. These 
commands are extracted at the digital demultiplexer 210 and sent to the controller 
260 which performs the appropriate action based on the commands. Some of these 
commands are explained below. 
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a. BEGIN INPUT EXTENDED 



The Begin Input command starts an input period during which the user may 
press one or more buttons to select his or her choice(s). The entry format of 
this command is set forth as follows: 

BEGIN INPUT {{N} {VALID KEYS} {DIFFERENT} {FEEDBACK} 

where: 

N The maximum number of keys that can be pressed. 

KEYS The valid keys that can be pressed. 

DIFFERENT Requires each key pressed to be different. 
FEEDBACK The feedback type provided to the viewer. 

b. BEGIN VIDEO CHOICE EXTENDED 

The Begin Video Choice begins an input period for disco mode. During disco 
mode, the video switches dynamically each time the viewer makes a 
selection. 

BEGIN_VIDEO_ CHOICE MODE [KEYS] {FEEDBACK [AUDIO]} 

The disco mode allows the viewer to change channels at will, while the 
OneShot mode allows only one change of channel. 

MODE DISCO/ONESHOT 

KEYS The valid keys that can be pressed. 

FEEDBACK The feedback supplied to the user for the key(s) which are 
pressed. 

The mode Disco allows the viewer to change channels at will, while the 
OneShot mode allows only one change of channel. 
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c BEGIN AUDIO CHOICE EXTENDED 

The Begin Audio Choice begins an input period for disco mode, 

BEGIN_AUDIO_ CHOICE MODE [KEYS] {FEEDBACK [AUDIO]} 

MODE DISCO/ONESHOT 

KEYS The valid keys that can be pressed. 

FEEDBACK The feedback supplied to the user for the key(s) which are 
pressed* 

d. MAP 

The Map command is used to map video or audio tracks to keys, for use in 
connection with the Begin Audio Choice and Begin Video Choice commands. 
If this command is omitted. Key 1 will map to Track or Channel 1, Key 2 to 
Track or Channel 2 etc. This command allows mapping any key to any 
channel. 

MAP KEYS [TRACKS [T1,...TN/T1-TN] CHANNELS [C1,...CN/C1-CN]] 

The map statement maps audio tracks or video channels to keys, to enable 
audio or video choice commands to effect changes to tracks other than the 
default tracks, which are that key 1 maps to track 1, key 2 to track 2, etc.. All 
the choice statements after a map statement will cause the tracks, video tracks 
or channels to be changed to those specified in the map statement. The 
acceptable numbers for the map command are from 1 to 8, for keys, video 
charmels and audio channels. 



Other commands include those which allow for the following applications: 

(1) viewer profiling, to enable the set top box 25 to "remember'' viewer preferences; 

(2) uploading viewer responses to a central location; (3) downloading of text and 
graphics, for display using the graphics chip of the set top box 25; (4) the ability of the 
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viewer to prepare his own video, based upon his selections of camera shots 100 and 
audio, which can be stored and replayed for the viewer. 

The interactive digital box 25 of the present invention enables seamless 
flicker-free transparent switching between the digital video signals. ''Seamless'' 
means that the switch from one video signal to another is user imperceptible. 
Because the video signals are running off the same clock, the interactive digital box 
25 is capable of providing a seamless digital switch from one video signal to another 
signal. The program clock reference necessary for the box to make this seamless 
switch is preferably embedded in the signal header. 

As shown in Figure 2, a CPU 260 is connected to an RF demodulator 200 and 
digital demultiplexer 210. The CPU 260 directs demodulation and demultiplexing of 
the proper channel and data stream to obtain the correct video signal. Seamless 
switching can occur with MPEG-2 compressed signals since there are points within 
the frame wherein seamless switching can occur. Preferably, switches occur at an "I" 
frame, assuming the use of MPEG-2 compression. The selected video signal is 
determined either by examination of the user's input from user interface 270 and /or 
any other information or criteria (such as personal profile information) stored in 
RAM/ROM 265. For example, the RAM/ROM 265 could store commands provided 
within the video signals as discussed in U.S. patent no. 4,602,279, and incorporated 
herein by reference. 
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The RF demodulator 200 demodulates data from the broadcast channel 
directed by the controller 260. After the data stream is demodulated, it passes 
through a forward error correction circuit 205 into a digital demultiplexer 210. The 
demultiplexer 210 is controlled by the controller 260 to provide a specific video 
signal out of a number of video signals which may be located within the data stream 
on the demodulated broadcast channel. The demultiplexed video signal is then 
decompressed and decoded by decompressor/decoder 215. The video signal is 
synchronized by a sync add circuit 220 and a sync generator 225. The video signal is 
then buffered by a video frame buffer 230. The buffered video signal is modulated 
by a modulator 245 into a NTSC compatible signal. Such a modulator is not 
necessary if the selected signal is sent to a digital television 195. 

By using a video frame buffer 230 and delaying the viewing of a given signal, 
enough time is allowed for the decompressor /decoder 215 to lock onto, decompress, 
convert to analog, and wait for the resultant vertical interval of a second video 
signal. For example, assume video signal A is currently being processed and 
transferred through the circuit shown in Figure 2 and displayed. Based upon a user 
selection, the controller 260 directs the digital demultiplexer 210 and RF 
demodulator 200 to switch to another video signal, video signal B. To accomplish 
this, the analog video from the first digital video signal, video signal A, complete 
with video sync, is fed into video frame buffer 230. This buffer 230 can hold the full 
video picture for "n" number of frames after which the signal is output to the 
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display. In effect, a delayed video signal A is viewed "n" number of frames after the 
signal has been received. When the user selects a different video path by means of 
pressing a button on a keypad or entry by other means, the controller 260 instructs 
the digital demultiplexer 210 to stop decoding signal A and lock onto signal B to 
begin decoding signal B instead of signal A. 

While this is happening, even though the decompressor/decoder 215 is no 
longer decompressing video signal A, the display is still showing video signal A 
because it is being read from the buffer 230. As soon as decompressing and decoding 
occurs, the controller 260 looks for the next vertical blanking interval (VBI) and 
instructs the video frame buffer 230 to switch to its input, rather than its buffered 
output at the occurrence of the VBI. 

Since the RF demodulator 200, forward error corrector 205, digital 
demultiplexer 210, and decompressor /decoder 215 require a certain time period to 
decompress and decode the video signal B frame from its data stream, the size of the 
buffer 230 has to be large enough so that this processing can take place without 
interruption during the switching of the video signals. If desired, the system may 
continue to use the buffer in anticipation of a future switch. By using the controller 
260 to manipulate the fill and empty rate of the buffer 230, the buffer 230 may be 
rapidly filled with video signal B frames and then after a period of time will be reset 
and ready to make another switch to another video in the same manner. The buffer 
230 may also be reset by skipping frames or providing a delay between sequential 
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frame outputs for a short time m order to fill the buffer 230. If a delay is used to 
maintain video signal or frame output while the buffer 230 is being filled, a slight 
distortion may occur for a brief amount of time. 

Because a first video signal is always displayed as the output of the buffer 230 
after the delay, the buffered video masks the acquisition and decoding of a second 
video signal. As long as the buffer 230 is large enough to keep the first video 
rurming while the second video is being decompressed and decoded, a seamless 
switch will occur. 

While the digital interactive box 25 of Figure 2 provides video interactivity, 
audio and/or graphics interactivity is also provided. For example, if, based on the 
viewer profile or viewer response to query, it is determined that the viewer's 
primary language is Spanish, then that viewer could obtain Spanish commentary to 
the football, soccer, etc. game. Alternatively, if a viewer has a favorite athlete, the 
audio can switch to an interview with the athlete during a segment of the broadcast. 
Multiple digital audio options forming a set of suitable responses to an interrogatory 
message can be sent as part of the composite digital signal. As set forth in U.S. 
patent no. 5,585,858, herein incorporated by reference, there are a number of 
different ways to effectively forward the necessary audio options for a given live 
event to the digital interactive box 25. With the present invention, it makes no 
difference how the audio options reach the digital interactive box 25, as long as they 
are available for selection and play at the appropriate times. 
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In Figure 2, the digital demultiplexer 210 extracts the digital audio signal(s) 
and forwards them to the audio switch 250. Additional audio options are available 
from the digital audio memory 255. At certain times during the program, the data 
codes will identify the selection of a particular audio option corresponding to 
previous user inputs. The controller 260 calls the appropriate audio options from 
internal memory 255 or directs the audio switch 250 to select a predetermined audio 
segment received as part of the received digital signal for passage to the RF 
modulator 245 for play to the subscriber. At the end of the audio segment time 
period as indicated by the data codes, the controller 260 instructs the audio switch 
250 to again pick up standard audio. 

The digital demultiplexer 210 sends the extracted graphics data or ACTV data 
codes to the controller 260. The controller 260 interprets the extracted data as either 
control data, including instructions for switching between video signals, audio 
signals, or graphics data for on-screen display. If the data is on-screen display data, 
the data is preferably prefixed by a command designating the data as on-screen 
display data, as opposed to control data. Further, the controller 260 also examines 
the control data for the occurrence of a header code designating the onset of a trigger 
point 500 in the program, explained below. 

Figure 3 shows an alternate, dual tuner embodiment for seamless switching 
between separate video signals. This embodiment presumes that two 6 MHz 
channels are used, each of which comprises compressed digital video and audio 
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streams. In this embodiment, the microprocessor 260 controls the selection of the 
RF charmel that is demodulated by RF demodulators 200A, 200B. The demodulated 
data streams enter the forward error correctors 205A, 205B. At the output of the 
forward error correctors 205A, 205B the data streams are transmitted to the input of 
the digital demultiplexers 210A, 210B. 

As with the RF demodulators 200A, 200B, the digital demultiplexers 210A, 
210B are controlled by the microprocessor 260. This configuration allows the 
microprocessor 260 to independently select two different individual time- 
multiplexed video signals on different channels and data streams. If all the video 
signals of an interactive program were contained on a single channel or data stream, 
it would only be necessary to have a single RF demodulator 200, forward error 
corrector 205, and digital demultiplexer 210 serially connected and feeding into the 
two digital video buffers 230A, 230B. 

Two data streams are provided from the digital demultiplexers 210A, 210B. 
One data stream carries video information pertaining to the video signal the user is 
currently viewing. The second data stream carries the video signal selected based on 
the user's previous and /or current interactive selections from the user interface 270, 
as determined by the microprocessor 260. 

The digital information on each of the two streams is buffered in digital video 
buffers 230 A, 230B. The buffered signals are then decompressed and converted into 
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analog signals by decompressors/decoders 215A, 215B which include digital to 
analog converters. The decompressors 215A, 215B are preferably MPEG-2 decoders. 

A local sync generator 225 is connected to sync add 220 A, 220B and frame sync 
380A, 380B circuits. Because both streams are synchronized based on signals from 
the same local sjmc generator 225, each stream becomes synchronized to the other. 
In particular, the signals on each stream are frame synchronized. 

A vertical blanking interval (VBI) switch 335 is connected to the 
microprocessor 260 so that the input may be switched during the vertical blanking 
interval of the current stream, resulting in a seamless switch to the viewer. 

The embodiment of Figure 3 operates as follows. Based on user responses 
and control codes, it is assumed that the microprocessor 260 determines that a 
switch from video signal A to video signal C should be performed. The RF 
demodulator 200A and digital demultiplexer 210A are processing the currently 
viewed video signal, video signal A, which is progressing through the upper branch 
components. A command is issued from the microprocessor 260 to the RF 
demodulator 200A, 200B commanding a switch to the channel and data stream on 
which video signal C is located. The microprocessor 260 also instructs the digital 
demultiplexer 210B to provide video signal C from the received data stream to 
digital video buffer 230B. 
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At this point, the upper RF demodulator 200A and digital demultiplexer 210A 
are still independently receiving and processing video signal A, which continues 
through the upper branch of the circuit. 

At a certain point, the digital decompressor/ decoder 215B in the lower branch 
will begin filling up with video signal C frames. After video signal C is 
decompressed and decoded, it is converted into analog. A local sync generator 225 
inserts both local sync and frame sync to video signal C via sync add circuit 220B and 
frame sync circuit 380B in order to synchronize it with the currently displayed video 
signal A, which is still being provided from the upper digital video buffer 230A. At 
the appropriate switch point, triggered by programming codes supplied with each 
video signal A and C, the microprocessor 260 directs the VBI switch 335 to switch in 
the vertical blanking interval from video A to video C, at which time video C will 
then seamlessly appear on the computer screen. 

Digital video buffers 230A, 230B may be used in the circuit of Figure 3, but are 
optional. However, in an alternative embodiment the buffers would be required to 
provide a seamless switch if the Figure 3 circuit was modified to incorporate a single 
RF demodulator 200, single forward error corrector 205, and single digital 
demultiplexer 210, each with a single input and single output. In this alternative 
embodiment, the circuit cannot independently receive and demultiplex two data 
streams on different frequency channels. One buffer 230A is used to store 
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previously received video signals, while the other buffer 230B quickly passes 
through the selected video signals. 

Based on the same assumptions above, video signal A is progressing through 
the upper branch of the circuit and it is desired to switch to video signal C. 
However, in this alternative embodiment, the digital video buffer 230A is providing 
maximum buffering to video signal A. 

Because it is desired to switch to video signal C, the microprocessor 260 directs 
the alternative circuit (containing a single RF receiver 200, single forward error 
corrector 205 and single digital demultiplexer 210 connected in serial), to receive and 
demultiplex the data stream on which video signal C is located, which may be 
different than that of video signal A. When video signal C is demultiplexed, the 
microprocessor 260 directs the digital video buffer 230 to provide minimum 
buffering of video signal C so that decompressor/decoder 215 may quickly 
decompress and decode the digital signals. After decompression and decoding, 
video signal C is synchronized with video signal A. At this time, video signal A is 
read for display from digital video buffer 230A. The upper digital video buffer 230A 
must be large enough to provide video frames for output during the time it takes 
the RF demodulator 200 and digital demultiplexer 210 to switch to video signal C 
and the time required for decompression, decoding, and synchronization of video 
signal C. 
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When video signal C is synchronized with video signal A, the 
microprocessor 260 directs VBI switch 335 to switch from video signal A to video 
signal C in the vertical blanking interval of video signal A, thereby providing a 
seamless and flicker-free switch* 

At this time, digital video buffer 230 will begin to utilize maximum buffering 
by altering its fill/empty rate as described above with respect to the Figure 3 
embodiment. When adequate buffering is achieved, a switch to another video 
signal may be performed in the same manner as described above. 

Another preferred embodiment is shown in Figure 4. This embodiment also 
includes an RF demodulator 200, a forward error corrector 205, and a digital 
demultiplexer 210. However, the circuitry differs along the rest of the chain to the 
television set or monitor. In this embodiment, a memory 475 is incorporated and 
connected to the output of the demultiplexer for storing the compressed composite 
digital video signal* The decompressor/decoder 215 is inserted at the output of the 
compressed memory. The decompressor/ decoder 215 decompresses the digital 
signal, converts the signal to analog and forwards the analog signal to the RF encode 
245 for transmission to the monitor. Once the composite compressed digital video 
signal is fed into the compressed memory 475, the microprocessor 260 directs a 
pointer to be placed somewhere along the compressed digital video signal. Based on 
the placement of the pointer, different frames and different segments of the 



29 



Docket No. 5038 



composite digital video signal will be read from memory 475 for decompression and 
decoding. 

The different video signals are distinguished from one another because they 
are labeled, preferably by headers. Assuming that video signal A has been selected 
for play on the monitor, the compressed digital memory 475 fills up with A frames. 
Assuming a switch to video signal C is desired, the microprocessor 260 directs the 
RF demodulator 200 and digital demultiplexer 210 to begin filling the compressed 
memory 475 with video C frames. The decoder 215 pointer begins to move down. 
As soon as a sufficient number of C frames have entered the compressed memory 
475, the pointer will then jump to the beginning of the C frames. The C frames are 
then output into the decompressor/decoder 215 where the digital frames are 
converted into an analog signal. 

The digital video is multiplexed in a series of easily identifiable packets. 
These packets may contain full compressed frames of video (I frames) or may 
include only the differences between full frames (B frames or P frames). 

To be able to reconstruct the full video images, the decompressor/decoder 215 
needs to have a minimum number of I, P and B frames. The decoder 215 needs only 
one I frame to decode an image. Conversely, two prior Anchor frames {'Ts" and 
"P's") are necessary to decode B frames. In order to decode P frames, the decoder 215 
only needs one Prior Anchor frame. When the microprocessor instructs the digital 
demultiplexer 210 to start sending packets from a different data stream there is no 
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way to be certain that the next packet will be an I packet needed for decoding the 
second video stream. To avoid a breakup of the video images, which would occur if 
the decompressor/ decoder 215 suddenly started receiving packets unrelated to the 
stream it was decoding, the microprocessor 260 starts to fill up the memory 475 with 
video signal C packets until it is determined that a full sequence of I, B and P frames 
are available. The decoder 215 should receive the last bit of the last B frame in a 
given, GOP (Group of Pictures) before the switch, in order to prevent glitches when 
decoding. Furthermore, the last B frame of the GOP must only be backward 
predicted, not forward predicted or bidirectional predicted. As soon as the valid 
sequence is in memory 475 the microprocessor 260 moves the memory read pointer 
to the start of a valid sequence of C video signal packets so that the 
decompressor/ decoder 215 can successfully decode the C signals. This results in a 
seamless switch from video signal A to video signal C. 

This embodiment requires a data channel for enabling a synchronous switch 
between a first video stream and a second video stream. This data channel 
comprises the ACTV codes which link together the different program elements and 
information segments on the different video signals. In addition, the data channel 
also comprises synchronization pulses and a time code to signify to the pointer the 
proper time to skip from a memory location representing one video signal to a 
memory location representing another video signal in order to enable a seamless 
switch. 
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The microprocessor 260 reads the data signal from the digital demultiplexer 
210 and communicates pertinent data to the sync add circuit 220, which is connected 
to sync generator 225. The microprocessor 260 is then able to synchronously 
communicate with the memory 475. 

The time code sent will identify the timing for one picture, as well as for 
multiple pictures, and will lock the different pictures together. This is done through 
the use of similar clocks at both the transmission end and the receiver. A time code 
is used in order to keep the two clocks at both the transmission and receive end 
synchronously connected to one another. Once the clocks at both ends are working 
synchronously, each of the multiplexed video streams must be synchronized to the 
clocks. In order to synchronize the multiplexed video stream to the clocks, each of 
the individual channels must be referenced to a common reference point and must 
be identified. 

In the preferred embodiment, a packet header would be incorporated into the 
transport layer of the MPEG signal to identify the various charmels. The packet 
header will also include information as to where to insert the vertical blanking 
interval. In MPEG, the vertical blanking interval is not transmitted from the 
headend. Therefore, the vertical blanking interval must be generated locally. The 
packet header eye will identify at what time the vertical blanking interval is in 
existence in order to effectuate a seamless switch between analog pictures. 
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In summary, the combination of clock and the information embedded in 
either the transport layer of MPEG or in a separate packet on a separate data channel 
effectuates the linking between each video signal and a corresponding time point. 
The data charmel also includes information designating when all the various video 
signals will be in synchronism with one another. It is at these points that the 
microprocessor 260 may direct the pointer to skip from one location to another 
location, at a time (such as during the VBI) when a seamless switch will result. 

D. Trigger Points 

Interactivity is further enhanced in the digital interactive embodiments 
through the application of trigger points 500 scattered at various predetermined 
times throughout the program, a timeline representation of which is shown in 
Figure 5. The trigger points 500 correspond to times when interactive events are 
scheduled to take place during the live sporting event 10. These interactive events 
could be the selection and playing of video, audio segments, the display of graphics 
or display of Web pages accessed from Internet Web sites 170. For example, when a 
viewer's favorite baseball player is at bat, graphics showing past or current 
performance statistics of the player can be overlaid on the screen while excerpts 
from an interview with the player can be played for the viewer. While the choice of 
particular video, audio or graphics is still dependent on viewer selections, the 
viewer selections in response to displayed graphical interrogatory messages are 
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preferably made during a period at the onset of the program or when a viewer first 
tunes into the program. Alternatively, interrogatories are not necessary if the 
switches are based on the viewer profile stored in memory 265. These viewer 
selections are then utilized as inputs to macros called up at later times during the 
program by the controller 260 upon the occurrence of the trigger points 500, 
identified to the interactive computer by unique codes embedded in the video 
signal. 

The trigger points 500 correspond to the times when the conventional 
program content can be altered and personalized for the viewers. The programmer 
can place the trigger points 500 at any time throughout the program. Since the 
trigger points 500 are unknown to the subscriber, the subscriber does not know 
when they will receive a personalized message. In other words, an interactive 
response can either immediately follow a corresponding user selection made to an 
interrogatory message or occur at a later time corresponding to a trigger point 500, or 
any combination of the two. Of course, timing of the interactive events should 
correspond to suitable times in the program where branching to interactive 
elements is sensible and does not clash with the program content of the 
conventional video still displayed on the television 165 or other display monitor. 

At the onset of a trigger point 500, the controller 260 will select one of several 
possible audio (or video or graphic display) responses for presentation to the 
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subscriber. As mentioned above and shown in figure 5, some of the responses may 
comprise a branch to either a video segment, graphics and/or audio segments. 

In combination with the use of trigger points 500, the present invention 
allows for the viewer to select certain options at the onset of the program to suit the 
viewers' preferences. For example, if the program broadcast is a live sports event 10, 
at an early trigger point 500, the viewer could be queried as to whether the viewer 
would prefer to receive audio in English, Spanish, French, or perhaps hear the local 
announcer instead of the network announcer. Upon the viewer selection, the CPU 
260 directs a branch to the appropriate interactive segment. 

Each trigger point 500 is identified preferably through the broadcast of ACTV 
codes sent as part of the composite interactive program signal. The codes preferably 
include, at a minimum, the following information: (1) header identifying the 
occurrence of a trigger point 500; (2) fimction ID (e.g., selection of audio or graphics 
responses, etc.); and (3) corresponding interrogatory message(s) or particular viewer 
characteristic or habit based on viewer profile. The first bit sequence simply 
identifies to the controller that a trigger point 500 is about to occur. The function ID 
designates the macro or other set of executable instructions for the controller 260 to 
read and interpret to obtain the desired result, e.g., a selected video and/or audio 
response. 

Upon extraction of the codes by the data decoder, the controller 260 reads and 
interprets the codes and calls from memory 265 a particular user selection(s) 
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designated by the trigger point 500 codes. The user selections correspond to 
subscriber answers to a series of interrogatory messages preferably presented at the 
beginning of the program. After obtaining the appropriate user selection(s), the 
controller 260 reads and performs the executable instructions using the user 
selection(s) as input(s) in the macro algorithm. The result of the algorithm is either 
a selected video stream, audio and /or selected graphics response. The video /audio 
response can be called from memory 265 if it is prestored, called from extemal data 
storage, or the controller 260 can command the switch to branch to the particular 
video audio stream if the response is broadcast concurrently with the trigger point 
500. After the selected video/audio response is played to the subscriber, the switch 
branches back to the standard program, shown at time t^ in Figure 5. 

As mentioned above, a series of interrogatory messages are preferably 
presented when the subscriber begins watching the interactive program. These 
interrogatory messages can be presented in any one of three ways. First, the 
interrogatory messages can be presented as graphics displays overlaid by the 
interactive computer workstation onto a video signal, wherein the graphics data is 
sent in the vertical blanking interval of the composite interactive signal, or 
alternatively stored on the hard disk or external storage. Second, the interrogatory 
messages are presented as graphics displays as discussed above, except the graphics 
data comes from local storage, external data storage (e.g., CD ROM, cartridge, etc.), or 
a combination of data in the VBI and data called from either local or extemal data 
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storage. Third, graphics data can be presented in the form of user templates stored at 
the interactive computer workstation. 

User selections corresponding to answers to the n successive interrogatory 
messages are received by the remote interface 270 at the beginning of the show, 
stored in memory 265 and used throughout the show at the appropriate trigger 
points 500 to subtlety change program content as the show progresses. Preferably, 
each interrogatory has a set of possible answers. Next to each possible answer will be 
some identifier corresponding to a label on a key on the user interface. The 
subscriber depresses the key corresponding to their answer selection. This selection 
is decoded by the remote interface 270 and controller 260, stored in memory 265, 
preferably RAM, and used later as required by an algorithm designated at a trigger 
point 500. 

E. Internet 

In addition to the central studio serving as a source of interactive option 
responses, the Internet can be used as a source of personalized information for 
interactive responses. 

As discussed above, the video programming is preferably created at a 
centralized location, i.e., the control studio 5 as shown in Figure 1, for distribution to 
subscribers in their homes. Referring to Figures 1 and 6, in a preferred method, the 
operator at the control studio 5 must designate certain Web pages to correspond to 
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one of the program optioris, such as audio and/or graphics options, using control PC 
135 and URL encoder 600. Instead of encoding the actual content of the options at 
the control studio 5, as with the audio, Web address identifiers, i.e.. Uniform 
Resource Locators (URLs) are encoded and sent as part of the data codes from the 
control PC 135 to the digital multiplexer 140. After multiplexing, the URLs are sent 
as part of the program signal 615, as described above. Preferably, the URLs, like the 
various audio and graphics options, have associated time stamps which indicate to 
the remote digital set top boxes 25 when, during the video program, to display the 
particular Web pages addressed by the URLs, the selection and display of which is 
preferably made as a function of viewer responses or viewer profile. 

Preferably, each digital set top box 25 has an Internet connection 160 created 
concurrently with the cable connection. The Internet connection 160 can be via 
high-speed line, RF, conventional modem. The digital set top box 25 has Internet 
access 160 via any of the current ASCII software mechanisms. In a preferred 
embodiment, in the interactive digital set top box 25, the digital demultiplexer 210 
extracts the URLs along with the other data codes. In an alternative embodiment, a 
local URL decoder 605 at the user site extracts the URLs. 

In a preferred embodiment, a JAVA enabled browser as well as specialized 
software for performing part of the method of the present invention are installed on 
the interactive digital set top box 25. The JAVA enabled browser allows the 
interactive digital set top box 25 to retrieve the Web pages and is preferred software. 
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since it is platform independent, and thus, enables efficient and flexible transfer of 
programs, images, etc., over the Internet. The specialized software acts as an 
interface between the video programming and the Internet functions of the present 
invention. The processor and software interprets these URLs and directs the JAVA 
enabled browser to retrieve the particular relevant Web pages, and synchronizes the 
retrieved Web pages to the video content for display on the television monitor 165 
at the appropriate times. 

In the present invention, the viewer also has the capability to link to a 
channel website at will. For example, if a viewer is interested in purchasing a 
product described in an advertisement, by merely clicking on a button on their 
remote 20, the producer's Website could be accessed by Internet connection 160 and 
displayed to the viewer. The viewer could then either obtain more information 
about the product or order the product, if desired. As described above, this 
application is possible by sending the URL associated with the producer's Website to 
the digital cable boxes 25 as part of the interactive program. Upon selection by the 
viewer, the web browser, located either in the digital set-top box 25 or externally in a 
connected PC 610, can retrieve the Web pages. The specialized software then 
synchronizes the Web pages for video display. 
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F. Memory 

The interactive digital set top box 25 of the present invention also has the 
advantage of remembering subscriber responses and using these responses in 
choosing a video/ audio response, and/or graphics interrogatory message, to present 
to the student. Memory branching is a technique of the present invention where 
the algorithm assembles video /audio responses and graphics interrogatory messages 
according to the current and previous user inputs. Memory branching is 
accomplished by linking video/audio streams and/or successive graphics 
interrogatory messages together in a logical relationship. In this scheme, the 
interactive digital set top box 25 contains logic (preferably, in the software algorithm) 
and memory 265 to store previous subscriber selections and to process these 
previous responses in the algorithm to control future video /audio stream selection, 
as well as future graphics message selection. 

G. Digital Viewer Profiles 

In a preferred embodiment, the interactive digital cable box 25 can have a 
"viewer profile" stored in its memory 265. Alternatively, the accumulated profile of 
viewer characteristics and/or habits can be stored at the control studio 5 or cable 
headend. If the profile statistics are accumulated at some central location, they can 
also be sent to each viewer's home. On the other hand, if accumulated in memory 
265 at each of the set top terminals 25, the data could be sent to the central location 
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for storage and dissemination by way of the digital back channel. Decisions 
regarding personalized advertising and viewing can then be made for a viewer or 
class of viewers based on the accumulated parameters. 

The "viewer profile" preferably contains characteristics of the particular 
viewer at that subscriber location, such as sex, hobbies, interests, etc. This viewer 
profile is created by having the viewer respond to a series of questions. 
Alternatively, the viewer profiles could be created at a control studio 5 and sent to 
the interactive digital cable box 25. This information is then used by the cable box 
software to create a compendiimi of the viewer's interests and preferences ~ i.e., a 
user profile. The stored user profile would be used in place of the question/ answer 
format, and thus, dictate the branches to interactive segments of interest to the 
viewer. 

Alternatively, the interactive cable box 25 can be programmed to create a user 
profile of each viewer based on the selections made during one of the interactive 
programs. If the profile shows that a particular viewer does not enjoy violence, the 
system can automatically branch to another video signal at the commencement of a 
fight in a sports program. Furthermore, such a user profile could be modified or 
enriched over time based on selections made during future interactive programs. 
For example, the 'memory' technique described above can be used to modify the 
user profile based on user response over time. 
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Event data is collected from the viewer when the following command is 
received and processed by the controller 260 in the interactive digital cable box 25: 

VIEWER PROFILE 

This command is used to enable events which can be utilized for prof ilin g 
during a show. The events indicated in this command are those profiling 
events which are possible during the show. The enabled events may be either 
selected by the viewer during the show, or may be automatically enabled 
based upon viewer selections. 

VIEWER_PROFILE EVENT1,...EVENT24 

Once the profile is created, the programming choices or interactive responses 
can be triggered based on the content of the viewer profile itself. For example, if the 
viewer profile suggests that the viewer is particularly interested in sports cars, a 
sports car commercial could be played for the viewer at a predetermined point in the 
program. As another application, if a viewer's profile indicates that the viewer is 
interested in cooking, whenever the viewer watches such a program, the user 
profile would trigger the interactive program to download recipes and either display 
such recipes on the screen or send the recipes to an attached printer. 

Viewer profile information can then be collected at the control studio 5 
through polling of the viewer cable boxes 25 for viewer selection data. Special 
polling software is loaded into the digital set top box 25 for performing the polling 
functions. Alternatively, the invention allows for the digital set top boxes 25 to send 
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back data on command from the control studio 5 or periodically. The command to 
initiate an upload of viewer profile data is as follows: 



UPLOAD EXTENDED 



This command is used to initiate the uploading of data to the central site. 



UPLOAD_EXrENDED UPLOAD_IDENTIFIER VARIABLE_NAME 
[PHONE_NUMBER] 



UPLOAD„IDENTIFIER 
VARIABLE_NAME 
PHONE NUMBER 



numeric constant identifying upload, 
name of variable to be uploaded 
string, that consists of numbers only, 
it can be omitted, if not needed. 



Regardless of whether a polling or periodic scheme is used, the statistics and other 
user profile information is preferably sent back to the control studio 5 by use of the 
back-channel. 

H. Applications 

The embodiments, described above, allow for several possible applications. 
For example, in a live sports event 10, one channel could carry the standard video 
channel, with other channels carrying different camera angles 100 and/or close-ups 
of particular players. Other potential video options include instant replay, 
highlights, player statistics via graphic overlays, etc. Graphics presenting statistical 
information on the players can be constantly updated using the Chyron system. 
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The provision of trigger points 500, explained above, can provide for seamless 
integration of such video options during the sporting event, based on either the 
viewer responses to interrogatories at the begirming of the program and /or on the 
digital viewer profile. 

Further, the viewer can become the director with the present invention. For 
example, the viewer can choose which camera angles 100 to emphasize. In a 
broadcast of golf, the viewer can direct whether they desire to follow a particular 
player from hole-to-hole, focus on one particularly difficult hole. In this marmer, 
the viewer can customize the sporting broadcast to meet his own interests. 

Further, the viewer can act as a director to create their own video. During a 
live concert 10, for example, the viewer selects various camera angles 100 at different 
times. These selections along with a time stamp, indicating the program time when 
each selection was made, are stored in memory 265. When the program is played 
back a second time, the processor 260 will automatically direct branching between 
the video channels according to the stored selections at the time stamp. In this 
marmer, a music video can be created by the viewer. 

The viewing experience can be further enlightening for the viewer by 
implementing games and contests during the live sporting event 10. For example, 
graphics overlays can be developed that query the viewer during the game. During 
a football broadcast, for example, viewers can be queried with such interrogatories as 
the following: 
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What will be the next play? (RUN/PASS/KICK); 
Will the offense get the first down?; 
Will they score on this possession?; 
Pick the halftime score; 
Who will win? 

Each viewer's responses can be sent back to the control studio 5 for tabulation 
of scores. Preferably, the responses are packaged at the digital cable box 25 and 
transmitted to the control studio 5 via the digital backchannel upon the UPLOAD 
EXTENDED command. Alternatively, tabulation of scores can take place at the 
digital cable box 25 through the utilization of certain software in memory 265. Each 
correct answer can correspond to a certain number of points. At the end of the 
game, the interactive program preferably presents a graphic showing the viewer 
point total. If desired, advertisers could present special gift certificates for excellent 
performance in such games. The provision of such certificates would occur by 
displaying a certain code that a viewer can take to a store to receive the gift. In this 
manner, viewer interests in sports events can be enhanced. 

Further, the viewer has the option with the present invention to block out 
viewing of certain events. For example, if the viewer is adverse to violence during 
a sporting event or other type program, the system can block out such options from 
the viewer with the following data commands: 

EVENT 



45 



Docket No. 5038 



This command is used to indicate occurrence of a certain event (e.g*, a fight 
breaking out during a football game). 
EVENT EVENT_NUMBER 
EVENT_NUMBER is a numeric constant, 

ON EVENT EXECUTE MACRO 
This command is used to implement certain actions (such as an automatic 
branch to another video channel in order to block out violent event, for 
example), as soon as the event arrives. 
ON_EVENT EVENT_NUMBER MACRO__NUMBER 
EVENT_NUMBER is a numeric constant. 
MACRO_NUMBER is a numeric constant, 

L Two-Way Configuration 

The live programming system of the present invention may be operated in a 
two way configuration, as illustrated in Figure 7. In this mode, the various video 
signals are processed as previously described, being digitized and compressed at the 
control studio 5. The signals are then sent to a central switching station, or headend 
30. 

In this embodiment, the switching between the various live digital signals is 
accomplished at the headend 30 rather than at the receiver. On the receive end, 
each digital set-top box 760 relays viewer selections back to the remotely located 
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switching station 30. Preferably, the viewer selections are relayed by way of the 
digital back channel 770. However, the viewer selections may be relayed to the 
switching station 30 by any conventional means, such as two-way cable television, 
telephone or microwave transmission. The switching station 30 receives the viewer 
selection and routes the desired signal to a transmitter 750 which conventionally 
transmits the desired video down the appropriate digital cable channel for the 
particular viewer. 

At the central switching station 30, a demultiplexer 710 demultiplexes the 
compressed signals and places each on a separate bus channel 725. A number of 
remote control interactive switches 730, 732, 734, 736 are cormected to the video 
signal bus 725. Based on the viewer selections, an algorithm stored in memory 265 
and under processor 260 control at the central switching station 30, a digital seamless 
switch is made and the selected video, audio and/or graphics are forwarded to the 
viewer home for display. 

Such a two-way embodiment could be implemented in a video dial tone or 
video server system. In such a system, only a single video channel 755 is necessary 
for each home. Once the viewer selection is received at the server site at the cable 
headend 30, a switch is made to the appropriate video stream and this stream is sent 
on the single channel 755 to the home. 

Alternatively, it may be desirable to transmit an interactive sporting event 
over a single telephone line. When the viewer enters a selection on their remote 
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20, a signal is sent by way of the telephone line to the central switching station 30 
which routes the desired signal of the interactive program over the user's telephone 
line so that a single link handles both the interactive choice being made at the 
receiver and the transmission of that choice from the headend 30 where the actual 
switching takes place in response to the interactive selection made at the receiver. 

The two-way link between the viewer and switching station 30 may be used 
for other purposes. For example, demographic data may be transferred from the 
viewer to the broadcast network for commercial purposes, such as targeted 
advertising, billing, or other commercial or non-commercial purposes. 

While the present invention has been described primarily with respect to live 
events, and in particular sporting events, it has equal potential for enhancing 
content in other program categories. A viewer can become their own director of a 
murder mystery or other drama. By entering responses to displayed questions at the 
initiation of or during the show, the program will branch to alternative 
video/ audio segments as a result of the user selections. In this manner, different 
viewers with different selections may end up with a different murderer at the 
conclusion of the broadcast. 
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Using the foregoing embodiments, methods and processes, the interactive 
multimedia computer maximizes personalized attention and interactivity to 
subscribers in their homes in real time. Although the present invention has been 
described in detail with respect to certain embodiments and examples, variations 
and modifications exist which are within the scope of the present invention as 
defined in the following claims. 
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CLAIMS 

We claim: 

1. A live interactive digital programming system, comprising: 

a viewer television reception system for receiving live interactive 
programming, the live interactive programming comprising a plurality of digitally 
compressed video, audio, branching codes and graphics signals, the reception system 
comprising: 

a viewer interface for receiving viewer entries; 

a microprocessor, connected to the viewer interface, for selecting one of 
the video and audio signals and directing a seamless switch to the selected 
video and audio signals at a predetermined time, the selection of the video 
and audio signals and the predetermined time of each selection a fimction of 
the branching codes and the received viewer entries; 

a demultiplexer, for demultiplexing the selected video and audio 
signals; 

a decompressor /decoder, connected to the demultiplexer for 
decompressing the demultiplexed selected video and audio signals; 
a means for displaying the selected video signal; and 
a means for playing the selected audio signal. 
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2. The live interactive digital programming system of claim 1, wherein the 
plurality of digitally compressed video signals corresponds to different 
predetermined camera angles of an event. 

3. The live interactive digital programming system of claim 1, wherein the 
microprocessor selects one of the graphics signals at a predetermined time, the 
selection of the graphics signal a function of the branching codes and the received 
viewer entries, and further comprising a means, connected to the microprocessor, 
for presenting the selected graphics signal on the display means. 

4. The live interactive digital programming system of claim 1, wherein the 
display means presents at least one interrogatory to the viewer, the content of the 
interrogatory involving program options, and the viewer entries correspond to 
collected entries from the viewer via the viewer interface in response to the 
interrogatories. 

5. A live interactive digital programming system, comprising: 

a viewer television reception system for receiving live interactive 
programming, the live interactive programming comprising a plurality of digitally 
compressed video, audio, branching codes and graphics signals, the reception system 
comprising: 
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memory, for storing a viewer profile; 

a microprocessor, connected to the viewer interface, for selecting one of 
the video and audio signals and directing a seamless switch to the selected 
video and audio signals at a predetermined time, the selection of the video 
and audio signals and the predetermined time of each selection a function of 
the branching codes and the stored viewer profile; 

a demultiplexer, for demultiplexing the selected video and audio 
signals; 

a decompressor/decoder, connected to the demultiplexer for 
decompressing the demultiplexed selected video and audio signals; 
a means for displaying the selected video signal; and 
a means for playing the selected audio signal. 

6» The live interactive digital programming system of claim 5, wherein the 
plurality of digitally compressed video signals correspond to different 
predetermined camera angles of an event. 
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7. The live interactive digital programming system of claim 5, wherein the 
microprocessor selects one of the graphics signals at a predetermined time, the 
selection of the graphics signal a function of the branching codes and the viewer 
profile, and further comprising a means, connected to the microprocessor, for 
presenting the selected graphics signal on the display means. 

8. A live interactive digital programming system, comprising: 

a viewer television reception system for receiving live interactive 
programming, the live interactive programming comprising a plurality of digitally 
compressed video, audio, branching codes, and one or more uniform resource 
locators specifying one or more Internet addresses of related Internet information 
segments obtained from Web sites on the Intemet, the reception system comprising: 
a viewer interface for receiving viewer entries; 

a means, connected to the viewer interface, for processing comprising: 

means for selecting one of the video and audio signals and 
directing a seamless switch to the selected video and audio signals at a 
predetermined time, the selection of the video and audio signals and 
the predetermined time of each selection a function of the branching 
codes and the received viewer entries; 

a means for decoding the uniform resource locators to 
determine the specified Intemet addresses; 
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' * * 

a means, connected to the decoding means, for retrieving the 
one or more Internet information segments residing at the determined 
Internet addresses; and 

a means for presenting the video and audio signals, and Internet 
information segments. 

9. The Hve interactive digital programming system of claim 8, further 
comprising: 

a demultiplexer, for demultiplexing the selected video and audio 
signals; and 

a decompressor/ decoder, connected to the demultiplexer, for 
decompressing the demultiplexed selected video and audio signals. 

10. The live interactive digital programming system of claim 8, wherein the 
plurality of digitally compressed video signals correspond to a different 
predetermined camera angle of an event. 

11. The live interactive digital programming system of claim 8, wherein the 
presenting means displays at least one interrogatory to the viewer, the content of the 
interrogatory involving program options, and the viewer entries correspond to 
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collected entries from the viewer via the viewer interface in response to the 
interrogatories. 

12. The live interactive digital programming system of claim 8, wherein the live 
interactive programming further comprises a plurality of graphics signals and the 
selecting means selects one of the graphics signals at a predetermined time, the 
selection of the graphics signal a function of the branching codes and the viewer 
profile, and further comprising a means, connected to the microprocessor, for 
presenting the selected graphics signal on the display means. 

13. A system for providing live interactive digital programming, comprising: 

a means for receiving video signals from a plurality of video cameras, one or 
more of the cameras relaying a different predetermined view of a live event; 

a means for producing one or more audio signals corresponding to the live 

event; 

a means for generating one or more graphics signals; 

at least one digital compression device, cormected to the receiving and 
producing means, for digitally compressing the video, graphics and audio signals; 

a means for processing, connected to the compression device, wherein the 
processing means creates a set of data commands which link together the various 
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audio, graphics and video signals, the data commands including branching commands; 

a digital multiplexer, connected to the digital compression device, for 
multiplexing the video, graphics and audio signals, and the data codes into a 
combined digital program stream; and 

a means for transmitting the combined digital program stream, 

14. A method for providing live interactive digital programming, comprising the 
steps of: 

obtaining video signals from a plurality of video cameras, one or more of the 
cameras relaying a different view of a live event; 

producing one or more audio signals corresponding to the live event; 

creating one or more graphics signals; 

receiving the video and audio signals in a control studio; 

digitally compressing the video, graphic and audio signals; 

producing a set of data codes corresponding to the programming, the data 
codes including branching commands; 

digitally multiplexing the video, graphics and audio signals, and the data 
codes into a combined digital program stream; 

transmitting the combined digital program stream; 

receiving the combined digital program stream at a receive site; 
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re-transmitting the combined digital program stream on a digital cable 
television distribution system; 

receiving the combined digital program stream at one or more viewer sites; 
gathering viewer specific information; 
processing the data commands; 

digitally demultiplexing the video and audio signals resulting in a first video 
and audio signal, the first output video and first audio signal selected based on the 
data commands and gathered viewer specific information; 

instructing the digital demultiplexer to commence demultiplexing a second 
video and second audio signal, the second video signal and second audio signal 
selected based on the data commands and gathered viewer specific information; 

seamlessly switching from the first to the second video signal; and 

displaying the second video signal on a screen, 

15. The method of claim 14, further comprising the steps of: 

creating a viewer profile with the gathered viewer specific information; 
wherein selecting the video and audio signals are based in part on the viewer 

profile. 
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16. The method of claim 14, wherein the step of gathering viewer specific 
information comprises the steps of: 

displaying at least one interrogatory to the viewer, the content of the 

interrogatory involving program options; 

collecting entries from the viewer in response to the interrogatories; and 
wherein the selection of video or audio signals is based in part on the 

collected viewer entries. 

17. A method for providing live interactive digital programming, comprising: 
receiving live interactive programming, the live interactive programming 

comprising a plurality of digitally compressed video, audio, branching codes, and 
one or more uniform resource locators specifying one or more Internet addresses of 
related Internet information segments obtained from Web sites on the Internet, the 
reception system comprising: 

obtaining viewer entries; 

selecting one of the video and audio signals and directing a seamless switch to 
the selected video and audio signals at a predetermined time, the selection of the 
video and audio signals and the predetermined time of each selection a function of 
the branching codes and the obtained viewer entries; 

decoding the uniform resource locators to determine the specified Internet 
addresses; 
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retrieving the one or more Internet information segments residing at the 
determined Internet addresses; and 

demultiplexing the selected video and audio signals; 

decompressing the demultiplexed selected video and audio signals; and 

presenting the video and audio signals, and Internet information segments. 
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ABSTRACT 

The present invention relates to an interactive digital system enabling 
viewers full and active participation in experiencing a live broadcast event. 
Particularly, the presentation of the live event is personalized for the viewer 
through the provision of various options, including multiple video streams, 
associated with different camera angles, for example, and integrated audio and 
graphics segments. Further, information obtained from related Web sites can be 
integrated into the live program. Various video and audio streams are collected 
from a live event and forwarded to a central control studio. Graphics are created at 
the central studio on a personal computer or chyron device. After receiving the 
video, audio and graphics signals, the signals are digitized and compressed in digital 
compressors. These signals are then combined with special data codes into a "digital 
package," and subsequently, transmitted over a cable distribution system. Once 
received at a viewer home, the signals are received and processed in an interactive 
digital cable box. Selections of the video, audio, graphics displays and/or Web pages 
can be made as a function of immediate viewer entries, or to interrogatory responses 
presented at the beginning or during the program, or based on a prestored viewer 
profile. Once a decision is made to switch from one video to another video option, 
the digital switch is performed seamlessly. The digital interactive system is based 
upon seamless branches which occur in the course of full-motion video. 
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